Multimodal Communication In Support of Coordinative Functions in Human-Machine Teams

The ongoing evolution of modern technology from reactive tool to powerful and independent agent has created problems that are related to breakdowns in humanautomation coordination. These breakdowns can be explained, in part, by the fact that machines can initiate actions on their own and without explicit operator consent but do not possess the communication skills that are required to know when and how to share information with operators concerning their intentions, actions, and limitations. One problem in particular is the extensive and increasing use of automation feedback that requires focal visual attention. Such feedback does not support task-sharing and effective attention allocation, especially in the context of unexpected changes and events. One possible solution to the problem is suggested by multiple-resource theory: the distribution of tasks and information across various sensory channels. In this manuscript, we will provide an overview of our recent series of simulator studies on the effectiveness of multimodal feedback for supporting early stages of attention management. Also, some of the many remaining challenges associated with supporting more complex coordinative functions in human-machine teams will be discussed.